開始之前¶
- https://www.sli.do/ : P082
- 程式碼同步
使用 Google Colaboratory¶
關於我們¶
- 由台大資訊系統訓練班的講師組成
- 希望以 Web Development、Data Science 作主軸定期辦聚會
- 訂閱 Pyradise 與 DataInPoint 的 Medium 專欄
- 追蹤 Pyradise 與 DataInPoint 的粉絲專頁
關於我¶
資料科學與推廣教育的愛好者，台大商研所畢，創辦 DataInPoint 致力為資料科學愛好者提供最棒的教學資源。任職過上海的外商新創團隊、台北的外商軟體公司與民營銀行，閒暇時喜歡長跑與乒乓球。
議程¶
- 動機
- 前情提要
- 視覺化套件簡介
- 簡易視覺化
folium視覺化
動機¶
2018-11-24 之後有感¶
- 同溫層崩潰
- 動起來，覺得應該做些什麼
- 感謝 AWS、讚嘆 AWS！
許多好的應用¶
- 2018最新！【九合一選舉互動地圖】
- 2018 台灣公投結果中的不合理數據
- ...etc.
有效的社群訊息傳播不再是文字論述¶
- 長輩圖
- Noun-Project 簡報圖
- 資料佐證的資訊圖表
我打算怎麼動起來：傳播技能¶
- 如何用程式幫助資料的擷取與整併
- 如何用程式製作視覺化
- 如何用 Adobe XD 或 Vectr 美化向量圖表
前情提要¶
投票所明細資料來源¶
預期產出¶
- 直轄市長、縣市長各投票所明細整併為一個長表格
- 直轄市議員、縣市議員各投票所明細整併為一個長表格
- 公投各投票所明細整併為一個寬表格
直轄市長、縣市長資料整併¶
- 判斷有幾個候選人、姓名與黨籍
- 選定必要欄位
- 給定欄位名
- 填補行政區缺失
- 刪除行政區得票數小計列
- 轉置
- 垂直合併
已經寫好的函數 get_tidy_df¶
https://github.com/yaojenkuo/cec-2018/blob/master/tidy_election_data.py
In [1]:
from tidy_election_data import get_tidy_df
city_ids = [100, 200, 300, 400, 500, 600]
county_ids = list(range(701, 715)) + [801, 802]
cities = ["台北市", "新北市", "桃園市", "台中市", "台南市", "高雄市"]
counties = ["新竹縣", "苗栗縣", "彰化縣", "南投縣", "雲林縣", "嘉義縣", "屏東縣", "宜蘭縣", "花蓮縣", "台東縣", "澎湖縣", "基隆市", "新竹市", "嘉義市", "金門縣", "連江縣"]
admin_areas = cities + counties
city_xls_file_urls = ["https://s3-ap-northeast-1.amazonaws.com/tw-election-2018/city-mayor/{}.xls".format(cid) for cid in city_ids]
county_xls_file_urls = ["https://s3-ap-northeast-1.amazonaws.com/tw-election-2018/county-mayor/{}.xls".format(cid) for cid in county_ids]
xls_file_urls = city_xls_file_urls + county_xls_file_urls
df_dict = dict()
for file_url, admin_area in zip(xls_file_urls, admin_areas):
tidy_df = get_tidy_df(file_url)
long_df = tidy_df[1]
long_df.insert(0, "admin_area", admin_area)
df_dict[admin_area] = long_df
In [2]:
import pandas as pd
mayors = pd.DataFrame()
for k in df_dict:
mayors = mayors.append(df_dict[k], ignore_index=True)
print(mayors.shape)
mayors.head()
mayors.to_csv("mayors.csv", index=False)
從 Colaboratory 上將檔案下載到本機¶
from google.colab import files
!ls
files.download("mayors.csv")
直接用 pandas 讀入¶
In [3]:
import pandas as pd
mayors = pd.read_csv("https://s3-ap-northeast-1.amazonaws.com/tw-election-2018/mayors.csv")
print(mayors.shape)
In [4]:
mayors.head()
Out[4]:
In [5]:
mayors.tail()
Out[5]:
In [6]:
mayors.info()
視覺化套件簡介¶
Python 的常用視覺化套件¶
- 靜態
matplotlib.pyplotpandasseaborn
- 互動（htmlWidget）
bokehplotly/dashfolium
matplotlib.pyplot¶
- 最為 Python 使用者所熟悉的低階視覺化套件
- 作圖的資料單位是 list 或 ndarray
- https://matplotlib.org/users/index.html
pandas¶
- 以 matplotlib 常用圖形為基礎包裝好的高階視覺化套件
- 作圖的資料單位是 DataFrame
- https://pandas.pydata.org/pandas-docs/stable/user_guide/visualization.html#
seaborn¶
- 以 matplotlib 為基礎包裝好的高階統計視覺化套件
- 以統計分析目的作為大分類
- 作圖的資料單位是 DataFrame
- https://seaborn.pydata.org/
folium¶
- Python 的互動式地圖視覺化套件
- 基於 Leaflet.js
- https://github.com/python-visualization/folium
簡易視覺化¶
matplotlib.pyplot¶
In [7]:
import pandas as pd
import matplotlib.pyplot as plt
from matplotlib.font_manager import FontProperties
myfont = FontProperties(fname="/System/Library/Fonts/STHeiti Light.ttc")
tp = mayors[mayors["admin_area"] == "台北市"]
grouped = tp.groupby(["number", "candidate"])
ttl_votes = pd.DataFrame(grouped["votes"].sum()).reset_index()
ttl_votes
Out[7]:
In [8]:
plt.barh(y=ttl_votes["number"].values, width=ttl_votes["votes"].values)
plt.yticks(ttl_votes["number"].values, ttl_votes["candidate"].values, fontproperties=myfont)
plt.title("台北市長得票數", fontproperties=myfont)
plt.show()
pandas¶
In [9]:
import pandas as pd
tp = mayors[mayors["admin_area"] == "台北市"]
grouped = tp.groupby(["number", "candidate"])
ttl_votes = pd.DataFrame(grouped["votes"].sum()).reset_index()
ttl_votes
Out[9]:
In [10]:
ax = ttl_votes.plot(x="candidate", y="votes", kind="barh")
ax.set_yticklabels(ttl_votes['candidate'], fontproperties=myfont)
plt.show()
seaborn¶
In [11]:
import pandas as pd
import seaborn as sns
tp = mayors[mayors["admin_area"] == "台北市"]
grouped = tp.groupby(["number", "candidate"])
ttl_votes = pd.DataFrame(grouped["votes"].sum()).reset_index()
ttl_votes
Out[11]:
In [12]:
sns.barplot(x="votes", y="number", data=ttl_votes, orient="h", ci=None)
plt.show()
In [13]:
import pandas as pd
import seaborn as sns
city_names = {
"台北市": "Taipei",
"新北市": "New Taipei",
"桃園市": "Taoyuan",
"台中市": "Taichung",
"台南市": "Tainan",
"高雄市": "Kaohsiung"
}
cities = mayors[mayors["admin_area"].isin(["台北市", "新北市", "桃園市", "台中市", "台南市", "高雄市"])]
grouped = cities.groupby(["admin_area", "number"])
ttl_votes = pd.DataFrame(grouped["votes"].sum()).reset_index()
ttl_votes["admin_area_en"] = ttl_votes["admin_area"].map(city_names)
ttl_votes
Out[13]:
In [14]:
g = sns.FacetGrid(ttl_votes, col="admin_area_en", col_wrap=3)
In [15]:
g = sns.FacetGrid(ttl_votes, col="admin_area_en", col_wrap=3, sharey=False, sharex=False, col_order=["Taipei", "New Taipei", "Taoyuan", "Taichung", "Tainan", "Kaohsiung"])
g.map(sns.barplot, "votes", "number", orient="h", order=None)
plt.show()
folium 視覺化¶
安裝¶
- 終端機
pip install --upgrade folium
#conda install folium -c conda-forge
- Google Colaboratory
!pip install --upgrade folium
Quickstart¶
In [16]:
import folium
m = folium.Map(location=[25.0389, 121.5682], zoom_start=15)
In [17]:
m
Out[17]:
添加標記（marker）¶
In [18]:
import folium
marker_loc = [25.0389, 121.5682]
tooltip = 'Click!'
m = folium.Map(location=marker_loc, zoom_start=15)
folium.Marker(marker_loc, popup='台灣亞馬遜網路服務有限公司', tooltip=tooltip).add_to(m)
Out[18]:
In [19]:
m
Out[19]:
先暸解 GeoJSON 的內容¶
In [20]:
import requests
geojson = "https://s3-ap-northeast-1.amazonaws.com/tw-election-2018/twCounty2010.geo.json"
r = requests.get(geojson)
geojson_as_dict = r.json()
print(geojson_as_dict.keys())
In [21]:
for i in range(len(geojson_as_dict["features"])):
print(geojson_as_dict["features"][i]["properties"])
In [22]:
import folium
m = folium.Map(location=[24, 121], zoom_start=7, tiles='Mapbox Bright')
geojson = "twCounty2010.geo.json"
folium.GeoJson(
geojson
).add_to(m)
Out[22]:
In [23]:
m
Out[23]: